Systems and methods for interferometric multifocus microscopy

ABSTRACT

A system to generate image representations includes a first objective that receives a first light beam emitted from a sample and a second objective that receives a second light beam emitted from the sample, where the first light beam and the second light beam have conjugate phase. The system also includes a first diffractive element to receive the first light beam and separate it into a first plurality of diffractive light beams that are spatially distinct, and a second diffractive element to receive the second light beam and separate it into a second plurality of diffractive light beams that are spatially distinct. The system further includes a detector that receives the first and second plurality of diffractive light beams. The first plurality of diffractive light beams and the second plurality of diffractive light beams are simultaneously directed and focused onto different portions of an image plane of the detector.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the priority benefit of U.S. Provisional PatentApp. No. 62/688,646 filed on Jun. 22, 2018, the entire disclosure ofwhich is incorporated by reference herein.

REFERENCE TO GOVERNMENT RIGHTS

This invention was made with government support under Grant No.IIS-1453192 awarded by the National Science Foundation (NSF). Grant No.N00014-15-1-2735 awarded by the NSF, and Grant No. DE-AC02-06CH11357awarded by the U.S. Department of Energy. The government has certainrights in the invention.

BACKGROUND

Fluorescence optical microscopy is an imaging tool widely used inmolecular and cell biology because of its non-invasive and highbiochemical labeling capabilities. A traditional fluorescence microscopesystem includes a light source, an excitation filter, a dichroic mirroror beam splitter, and an emission filter. The light source can be in theform of an arc lamp, a vapor lamp, light-emitting diodes (LEDs), alaser, etc. A specimen to be imaged is illuminated with light from thelight source, at least a portion of which is absorbed by fluorophores inthe specimen. The excitation filter is used to direct one or moredesired wavelengths of light from the light source onto the specimen.Upon absorption of the light by the fluorophores, the fluorophores emitlight which typically has a longer wavelength than the absorbed light.The emission filter is used to separate the emitted light (i.e., fromthe fluorophores) from that of the light source. The emitted light isthen processed to develop an image of the specimen.

SUMMARY

Described herein is an interferometric multifocus microscopy (iMFM)imaging system that is able to achieve axial super resolution in asingle shot. In an illustrative embodiment, the proposed iMFM imagingsystem may include a first objective that receives a first light beamemitted from a sample being imaged and a second objective that receivesa second light beam emitted from the sample being imaged. In someembodiments, the first objective and the second objective are opposinglypositioned (i.e., relative to the sample), and the first light beam andthe second light beam have conjugate phase. Additionally, the firstlight beam and the second light beam can include light from differentdepth planes within the object.

The system may also include a first diffractive element that receivesthe first light beam passing through the first objective and thatseparates the received first light beam into a first plurality ofdiffractive light beams that are spatially distinct from each other. Insome embodiments, each of the first plurality of diffractive light beamsincludes light from one of the different depth planes within the object.The system may also include a second diffractive element that receivesthe second light beam passing through the second objective and thatseparates the received second light beam into a second plurality ofdiffractive light beams that are spatially distinct from each other.Each of the second plurality of diffractive light beams can also includelight from one of the different depth planes within the object.

The system can also include a detector that receives the first pluralityof diffractive light beams and the second plurality of diffractive lightbeams. In some embodiments, each of the first plurality of diffractivelight beams and the second plurality of diffractive light beams isdirected and focused onto a different portion of an image plane of thedetector simultaneously. As used herein, simultaneously refers to anevent that happens at substantially the same time (i.e., the events maypotentially occur fractions of a second apart). In some embodiments, thesystem may further include one or more optical elements that direct andfocus each of the first plurality of diffractive light beams and thesecond plurality of diffractive light beams onto the different portionsof the image plane of the detector at substantially the same time.

In some embodiments, the system can also include an image processingmodule that generates, from the directed first plurality of diffractivelight beams and the directed second plurality of diffractive lightbeams, a three-dimensional image representation of the sample. In someembodiments, the three-dimensional representation of the object includesa combination of a plurality of two-dimensional images, where each ofthe one or more two-dimensional images corresponds to one of thedifferent depth planes within the sample.

In some embodiments, the first diffractive element and the seconddiffractive element, which can be identical, are positioned in theFourier plane of the imaging system. Each of the first diffractiveelement and the second diffractive element can also include amulti-focus grating (MFG). In some embodiments, the multi-focus gratingincludes a grating pattern having a geometrical distortion. The gratingpattern can be a binary phase-only diffraction grating pattern or amulti-phase grating pattern. In an illustrative embodiment, themulti-focus grating focuses the first plurality of diffractive lightbeams and the second plurality of diffractive light beams by applying aphase shift that is equal to but opposite from a depth-induced phaseerror that is present on the light that emerges from out-of-focus planesin the sample. The phase shift can be provided by the geometricaldistortion in the grating pattern of the multi-focus grating.Additionally, the multi-focus grating can apply the phase shift to afirst wavefront of the light of the first plurality of diffractive lightbeams and to a second wavefront of the light of the second plurality ofdiffractive light beams, where the first wavefront and the secondwavefront have conjugate phase.

Also described herein are methods for generating an image representationof an object. The method can include receiving a first light beampassing through a first objective of an imaging system and a secondlight beam passing through a second objective of the imaging system. Insome embodiments, the first objective and the second objective areopposingly arranged relative to one another. The first light beam andthe second light beam are emitted from a sample (or object) beingimaged, and can have conjugate phase. Each of the first light beam andthe second light beam can also include light from different depth planeswithin the sample.

The method can also include generating, based on the received firstlight beam and by a first diffractive element, a first plurality ofdiffractive light beams that are spatially distinct from each other. Insome embodiments, each of the first plurality of diffractive light beamsincludes light from one of the different depth planes within the object.The method may also include generating, based on the received secondlight beam and by a second diffractive element, a second plurality ofdiffractive light beams that are spatially distinct from each other. Insome embodiments, each of the second plurality of diffractive lightbeams includes light from one of the different depth planes within theobject.

The method may also include directing and focusing each of the firstplurality of diffractive light beams and the second plurality ofdiffractive light beams onto a different portion of an image plane of adetector simultaneously. The method may further include generating, fromthe directed first plurality of diffractive light beams and the directedsecond plurality of diffractive light beams, a three-dimensional imagerepresentation of the sample. In some embodiments, the three-dimensionalrepresentation of the sample includes one or more two-dimensionalimages, each of which corresponds to one of the different depth planeswithin the sample.

Another illustrative system to generate image representations includes afirst objective that receives a first light beam emitted from a sampleand a second objective that receives a second light beam emitted fromthe sample, where the first light beam and the second light beam haveconjugate phase. The system also includes a first diffractive element toreceive the first light beam and separate it into a first plurality ofdiffractive light beams that are spatially distinct, and a seconddiffractive element to receive the second light beam and separate itinto a second plurality of diffractive light beams that are spatiallydistinct. The system further includes a detector that receives the firstand second plurality of diffractive light beams. The first plurality ofdiffractive light beams and the second plurality of diffractive lightbeams are simultaneously directed and focused onto different portions ofan image plane of the detector.

Another illustrative method for generating image representationsincludes receiving, by a first objective in a first arm of an opticalsubsystem, a first light beam emitted from a sample. The method alsoincludes receiving, by a second objective in a second arm of the opticalsubsystem, a second light beam emitted from the sample, where the firstlight beam and the second light beam have conjugate phase. The methodalso includes separating, by a first diffractive element in a Fourierplane of the first arm of the optical subsystem, the first light beaminto a first plurality of diffractive light beams that are spatiallydistinct from one other. The method also includes separating, by asecond diffractive element in a Fourier plane of the second arm of theoptical subsystem, the second light beam into a second plurality ofdiffractive light beams that are spatially distinct from one other. Themethod further includes receiving, by a detector, the first plurality ofdiffractive light beams and the second plurality of diffractive lightbeams such that the first plurality of diffractive light beams and thesecond plurality of diffractive light beams are simultaneously directedand focused onto different portions of an image plane of the detector.

BRIEF DESCRIPTION OF DRAWINGS

The foregoing and other objects, features, and advantages of the presentdisclosure set forth herein will be apparent from the followingdescription of particular embodiments of those inventive concepts, asillustrated in the accompanying drawings. Also, in the drawings, thelike reference characters refer to the same parts throughout thedifferent views. The drawings depict only typical embodiments of thepresent disclosure and, therefore, are not to be considered limiting inscope.

FIG. 1 is a block diagram of an interferometric multifocus microscopy(iMFM) system for three-dimensional imaging in accordance with anillustrative embodiment.

FIG. 2A is a detailed diagram of the iNIFNI imaging system of FIG. 1 inaccordance with an illustrative embodiment.

FIG. 2B depicts an upper arm focal stack of a second diffractive elementof the iMFM imaging system in accordance with an illustrativeembodiment.

FIG. 2C depicts a lower arm focal stack of a first diffractive elementof the iMFM imaging system in accordance with an illustrativeembodiment.

FIG. 2D depicts an interferometric focal stack on the detector inaccordance with an illustrative embodiment.

FIG. 3 is a flow diagram depicting operations performed by an iMFMimaging system in accordance with an illustrative embodiment.

FIG. 4A depicts the first multi-focus grating with focal step Δz=250 nmin accordance with an illustrative embodiment.

FIG. 4B depicts the second multi-focus grating with a focal step ofΔz=−250 nm in accordance with an illustrative embodiment.

FIG. 5A depicts an xy cut, an xz cut, a yz cut, and an axial profile ofeach tile point spread function (PSF) for an MFM monochratnatic PSF inaccordance with an illustrative embodiment.

FIG. 5B depicts an xy cut, an xz cut, a yz cut, and an axial profile ofeach tile PSF for an iMFM monochramatic PSF in accordance with anillustrative embodiment.

FIG. 5C depicts an xy cut, an xz cut, a yz cut, and an axial profile ofeach tile PSF for an MFM polychratnatic PSF in accordance with anillustrative embodiment.

FIG. 5D depicts an xy cut, an xz cut, a yz cut, and an axial profile ofeach tile PSF for an iMFM polychromatic PSF in accordance with anillustrative embodiment.

FIG. 6A depicts xz cuts of the square of the z-derivative for MEMmonochromatic PSF (left), MFM polychromatic PSF (middle left), iMFMmonochromatic PSF (middle right), and iMFM polychromatic PSF (right) inaccordance with an illustrative embodiment.

FIG. 6B depicts xz cuts of the combined z-derivative summed over 3×3tiles in FIG. 6A in accordacne with an illustrative embodiment.

FIG. 7A depicts theoretical axial localization precision σ_(z) forabberated and unaberrated detection schemes using the proposed iMFM andabberated and unaberrated detection schemes using an MFM system inaccordance with an illustrative embodiment.

FIG. 7B depicts the mean squared error of the z position determinedduring 50 simulated localizations per individual axial value for MFM andiMFM and corresponding theoretical predictions in accordance with anillustrative embodiment.

FIG. 8 depicts an example of determining the initial axial position andrange of a single point for an MILE localization algorithm in iMFM inaccordance with an illustrative embodiment.

FIG. 9A depicts a histogram of axial residuals between ground truth andMFM recovery in accordacne with an illustrative embodiment.

FIG. 9B depicts a historgram of axial residuals between ground truth andiMFM recovery in accordance with an illustrative embodiment.

FIG. 10A depicts a 3D extended test object in accordance with anillustrative embodiment.

FIG. 10B depicts recovery of the 3D extended test object using MFM inaccordance with an illustrative embodiment.

FIG. 10C depicts recovery of the 3D extended test object usingunaberrated iMFM in accordance with an illustrative embodiment.

FIG. 10D depicts recovery of the 3D extended test object using aberratediMFM in the presence of chromatic aberrations in accordance with anillustrativ embodiment.

FIG. 11 is a block diagram illustrating a programmed computer system foran interferometric multifocus microscopy imaging system in accordancewith an illustrative embodiment.

DETAILED DESCRIPTION

Traditional fluorescence optical microscopy systems have fundamentallimitations. First, in many instances, fluorescence optical microscopyis too slow to capture three-dimensional (3D) dynamic events, due to thelong acquisition time that it takes to sequentially z scan the focalplane by moving either the object stage or objective lens. Second, theaxial spatial resolution (roughly 500-700 nm) substantially lower thanthe lateral spatial resolution (roughly 200-300 nm) because of thelimited collecting angles of an objective lens. In the single objectiveconfiguration, the 3D intensity point spread function (PSF) features anelongated focal spot along the optical axis, and its optical transferfunction (OTF) suffers from a ‘missing cone’ problem along the axialdirection, Which poses a particular challenge in 3D isotropic microscopyimaging.

To overcome the 3D imaging speed limitation, Multifocal Plane Microscopy(MUM) can be used to allow simultaneous acquisition of multiple focalplanes in a single exposure time of the cameras. In some systems, thismethodology was implemented by using multiple beam splitters andcameras. In such a scenerio, each camera is placed at a specificdistance from the tube lens to enable the capture of images of adistinct focal plane within the sample. Arranging the cameras in such amanner typically only allows for the imaging of a limited number ofdistinct planes (commonly 4 planes) and becomes increasingly bulky ifmore planes are imaged since it utilizes one camera per focal plane.Another approach, known as Multifocus Microscopy (WFM), utilizes adistorted grating in the Fourier plane to image 9 or 25 focal planes ona single camera. The grating is designed to both diffract the light andalso focus different focal depths into different diffraction orders,producing laterally shifted images of multiple focal depths on a 2Dcamera plane at the cost of the lateral field of view.

Techniques such as MUM and MFM have attracted significant interest inapplications that involve the investigation of 3D dynamic samples.Examples include the tracking of a single molecule in three dimensionsor the detection of a dynamic process in various samples, such as thicksamples. However, like a conventional focal scanning microscope, bothMUM and MFM suffer from the anisotropic 3D resolution problem due to thelimited collecting angles of a single objective lens. For instance, inMUM single-particle tracking, the axial localization accuracy isrelatively worse than the lateral one.

Described herein is a high speed and efficient imaging system thatprovides a higher axial resolution and hence isotropic 3D resolution ina single exposure. Aspects of the present disclosure involve aninterferometric multifocus microscopy (iMFM) imaging system thatprovides significantly improved axial and hence isotropic 3D resolutionwith a single shot. The iMFM imaging system employs two specialdiffractive optical elements (DOEs) in the Fourier planes and twoopposingly arranged objectives. Also described herein are simulationresults for the iMFM imaging system. The simulation results illustratethat employing two DOEs enables the iMFM to ensure that the conjugatespherical wavefronts, emitted from the same molecule, are diffractedinto the same tile on the detector, and hence interfere effectivelyafter passing through two lenses. Both monochromatic and polychromaticpoint spread functions (PSFs) of the iMFM imaging system are simulated,and the image formation model is provided. The simulation results alsoillustrate that the iMFM imaging system is capable of recording multipleinterferometric focal planes simultaneously in a single shot, whichcontains axial super-resolution information.

In other aspects, the present disclosure demonstrates variousapplications of the iMFM imaging system, including single moleculetracking and 3D extended object recovery. In addition, the presentdisclosure provides a calculation of the Fisher information matrix (FIM)and the Cramér-Rao lower bound (CRLB) of iMFM PSFs. The disclosure alsodescribes a method to determine an initial axial position of a moleculeto improve the convergence of a maximum likelihood estimation (MLE)localization algorithm in iMFM. The disclosure further demonstrates,both theoretically and numerically, that isotropic 3D nanoscopiclocalization accuracy is achievable with an axial imaging range of 2micrometers (μm) when tracking a fluorescent molecule in threedimensions. Additionally, it is shown that diffraction limited axialresolution can be improved by at least three times in 3D extended objectrecovery with a single exposure by iMFM, which significantly increasesthe speed of the acquisition process of conventional dual-objectiveinterferometric microscopes.

FIG. 1 is a block diagram of an interferometric multifocus microscopy(iMFM) system 100 for three-dimensional imaging in accordance with anillustrative embodiment. The system 100 includes an optical subsystem110, a detector subsystem 130, and a processing subsystem 150.Alternatively, the system may include fewer, additional, and/ordifferent components. The optical subsystem 110 can include one or morelight sources. The one or more light sources may be tunable and capableof emitting lights with a wide range of frequencies. The opticalsubsystem 110 can also include one or more objectives and one or moreoptical elements (e.g., mirrors, tube lenses, etc.) to direct or focuslight passing through the one or more objectives. In an illustrativeembodiment, the system 100 utilizes a dual-objective detection schemeconstituted by two detection arms, each of which includes an objective.Each detection arm of the optical subsystem 110 may also include adiffractive element such as a multi-focus grating (MFG) that separateslight into spatially distinct diffractive orders (or diffractive lightbeams). The optical subsystem 110 can further include additional opticalelements (e.g., mirrors, lenses, beam splitters, etc.) that direct lightpassing through the diffractive elements onto an image plane of adetector in the detector subsystem 130. The readouts from the detectorsubsystem 130 are processed and reconstructed by the processingsubsystem 150 into an image representation of the object. The optical,detector, and processing components of the system 100 are described inmore detail below.

FIG. 2A is a detailed diagram of the iMFM imaging system 100 inaccordance with an illustrative embodiment. The optical subsystem 110 ofthe iMFM imaging system includes a light source 200 that is configuredto emit light. The light source 200, which can be one or more lightsources, can be a laser source, a light-emitting diode source, an arclamp, a vapor lamp, etc. The light from the light source 200 is directedto a dichroic mirror 202 that reflects the light toward a sample 208being imaged. A first objective 204 and a second objective 206 arepositioned on the bottom and top of the sample 208, respectively, suchthat the objectives are opposingly positioned about the sample 208. Inalternative embodiments, the objectives may be positioned differentlyrelative to the sample 208 (e.g., on the sides, etc.). The firstobjective 204 receives a first light beam 209 emitted from the sample208 and the second objective 206 receives a second light beam 211emitted from the sample 208 in response to the light from the lightsource 200. In an illustrative embodiment, the first light beam 209 andthe second light beam 211 have conjugate phase relative to one another.In some embodiments, each of the first light beam 209 and the secondlight beam 211 includes light from different depth planes within thesample 208.

The first objective 204, along with a series of mirrors, lenses, and adiffractive element, is part of a lower arm of the optical subsystem110. The first light beam 209 received by the first objective 204travels back through the dichroic mirror 202, reflects off a mirror 210and through a tube lens 212 such that an intermediate image 214 isformed in the lower arm. The first light beam 209 travels through a lens216 and into a Fourier plane of the lower arm where a diffractiveelement 218 is positioned. The first light beam 209 travels through thediffractive element 218, which separates the received first light beam209 into a first plurality of diffractive light beams that are spatiallydistinct from one other. In an illustrative embodiment, each of thefirst plurality of diffractive light beams includes light from adistinct depth plane within the sample 208. The inset to FIG. 2A depictsdifferent depth planes of the sample 208 in the z-direction andseparated from one another by Δz. The first plurality of diffractivelight beams reflect off of a mirror 220, through a lens 222, off of amirror 224, through a beam splitter/combiner 226, through a lens 228,through a lens 230, and into the detector subsystem 130. The dichroicmirror 202, the mirrors 210, 220, 224, the tube lens 212, the lenses216, 222, and the diffractive element 218 form the lower arm of theoptical subsystem 110. In alternative embodiments, a different number,position, and/or combination of mirrors, lenses, and tube lenses may beused to form the lower arm.

The second objective 206, along with a series of mirrors, lenses, and adiffractive element, is part of an upper arm of the optical subsystem110. The second light beam 211 receives by the second objective 206reflects off of a mirror 232, off of a mirror 234, and through a tubelens 236 such that an intermediate image 238 is formed in the upper arm.The second light beam 211 travels through a lens 240 and into a Fourierplane of the upper arm where a diffractive element 242 is positioned.The second light beam 211 travels through the diffractive element 242,which separates the received second light beam 211 into a secondplurality of diffractive light beams that are spatially distinct fromone other. In an illustrative embodiment, each of the second pluralityof diffractive light beams includes light from a distinct depth planewithin the sample 208. The second plurality of diffractive light beamsreflect off of a mirror 244, through a lens 246, through the beamsplitter/combiner 226, through the lens 228, through the lens 230, andinto the detector subsystem 130. The second objective 206, the mirrors232, 234, 246, the tube lens 236, the lenses 240, 246, and thediffractive element 242 form the upper rm of the optical subsystem 110.In alternative embodiments, a different number, position, and/orcombination of mirrors, lenses, and tube lenses may be used to form theupper arm.

The first diffractive element 218 and the second diffractive element 242can be multi-focus gratings (MFGs) that are identical or substantiallyidentical to one another (e.g., they may induce opposite phase shifts,but otherwise be the same). In some embodiments, the multi-focus grating(MFG) includes a grating pattern having a geometrical distortion. Forexample, the grating pattern can include a binary phase-only diffractiongrating or a multi-phase grating. In such embodiments, the multi-focusgratings focus the first plurality of diffractive light beams and thesecond plurality of diffractive light beams by applying a phase shiftthat is equal to but opposite from a depth-induced phase error that ispresent on the light that emerges from out-of-focus planes in theobject. The phase shift can be provided by the geometrical distortion inthe grating pattern of the multi-focus grating. In some embodiments, themulti-focus grating applies the phase shift to a first wavefront of thelight of the first plurality of diffractive light beams and to a secondwavefront of the light of the second plurality of diffractive lightbeams, where the first wavefront and the second wavefront have conjugatephase.

As discussed above, the detector subsystem 130 receives the firstplurality of diffractive light beams and the second plurality ofdiffractive light beams from the optical subsystem 110. In anillustrative embodiment, the detector subsystem 130 includes anelectron-multiplying charge-coupled device (EMCCD) detector.Alternatively, a different type of detector may be used. In anotherillustrative embodiment, each of the first plurality of diffractivelight beams and the second plurality of diffractive light beams isdirected and focused onto a different portion of an image plane of thedetector at substantially the same time. Various optical components ofthe optical subsystem 110 (e.g., elements 220, 222, 224, 226, 228, 230,244, and 246) work together to direct and focus each of the firstplurality of diffractive light beams and the second plurality ofdiffractive light beams onto the different portion of the image plane ofthe detector at substantially the same time. FIG. 2B depicts an upperarm focal stack of the second diffractive element 242 in accordance withan illustrative embodiment. FIG. 2C depicts a lower arm focal stack ofthe first diffractive element 218 in accordance with an illustrativeembodiment. FIG. 2D depicts an interferometric focal stack on thedetector in accordance with an illustrative embodiment.

Information from the detector subsystem 130 is provided to theprocessing subsystem 150. In an illustrative embodiment, the processingsubsystem 150 can be local to the detector subsystem 130 and theinformation can be provided through a direct wired or wirelessconnection. Alternatively, the processing subsystem 150 may be remote tothe detector subsystem 130 and the information can be provided through anetwork such as the Internet. The processing subsystem 150 can includeone or more transceivers to receive the data, one or more memories tostore the data, one or more interfaces to allow user interaction and thecontrol, and one or more processors to analyze the data and form the 3Dimage representation(s) of the sample 208. In some embodiments, thethree-dimensional representation of the sample 208 includes one or moretwo-dimensional images, where each of the one or more two-dimensionalimages corresponds to one of the different depth planes within thesample 208. An example processing subsystem is described in detail withreference to FIG. 11.

An alternative system to that depicted in FIG. 2A includes adual-objective image interference microscopy (I²M) with an MFM stage. Insuch a design, a regular I²M configuration is augmented by a 4f system(e.g., the relay lens system formed by the lens 228 and the lens 230)with a single multi-focus grating (MFG) in its Fourier plane (i.e.,positioned between the lens 228 and the lens 230). In such a system, theemitted light from the same fluorescent point simultaneously enters boththe first and second objectives with the conjugate wavefronts, and thenis coherently added to form the interference pattern on a common beamsplitter (BS) plane, as in I²M. However, when these two conjugatewavefronts emitted from the same fluorescent point propagate through thesame MFG, they will be diffracted and focused into the positive and thenegative diffraction orders on the detector. In such a configuration,the MFG incorrectly interferes the top-left focal plane from the top armof the interferometer with the top-left focal plane from the bottom armof the interferometer, and the top-middle focal plane from the top armof interferometer with the top-middle focal plane from the bottom arm ofinterferometer, and so on, until the bottom-right focal plane of the toparm is interferometrically combined with the bottom-right focal plane ofthe bottom arm. Therefore, such an I²M+MFM configuration, the MFG mayprohibit the correct interference to occur on the detector.

To overcome the above problem in such an I²M+MFM system, the proposediMFM imaging system 100 of FIG. 2A employs two MFGs that are placed inthe Fourier planes, as shown. This iMFM configuration features adual-objective detection scheme with an additional opposing objectivelens. Each detection arm is similar to a conventional MFM, whichincludes an objective lens, a tube lens, and a 4f system (relay lenssystems of the lenses 240, 246 in the upper arm and the lenses 216, 222in the lower arm) with an MFG in the Fourier plane of each arm.According to MFM principle, each objective lens images multiple focalplanes into an array of differently focused tiles on the detector withina single exposure of time.

One of the important differences between iMFM (shown in FIG. 2A) and anI²M+MFM is that, in the iMFM imaging system, the emitted light is firstsplit into multiple tiles and then coherently added on a common beamsplitter/combiner plane. Hence, if two MFGs of opposite focal shifts areemployed in the dual detection arms, the conjugate wavefronts from thesame fluorescent point will be diffracted and focused on the same tile,and therefore interfere on the same area of the beam splitter/combinerand the detector. That is, the differing MFGs in each arm place focalplanes at the sample place onto the same tile on the detector.Therefore, the proposed iMFM imaging system is capable of producing afocal stack of interferometric 2D images simultaneously on a singledetector with a single exposure without focal scanning.

FIG. 3 is a flow diagram depicting operations performed by an iMFMimaging system in accordance with an illustrative embodiment. Inalternative embodiments, fewer, additional, and/or different operationsmay be performed. Additionally, the use of a flow diagram is not meantto be limiting with respect to the order of operations performed. In anoperation 300, a first objective of the system receives a first lightbeam that is emitted from a sample. In an operation 305, a secondobjective of the system receives a second light beam that is emittedfrom the sample. In an illustrative embodiment, the first light beam andthe second light beam are emitted from the sample responsive to lightdirected at the sample from one or more light sources. In anotherillustrative embodiment, the first objective and the second objectiveare positoined opposite to one another, and the first and second lightbeams include light from different depth planes within the object.

In an operation 310, the first light beam is passed through a firstdiffractive element to generate a first plurality of diffractive lightbeams. In an operation 315, the second light beam is passed through asecond diffractive element to generate a second plurality of diffractivelight beams. In an illustrative embodiment, the diffractive elements aremulti-focus gratings having grating patterns with a geometricaldistortion such that each of the first plurality of diffractive lightbeams has a different phase and each of the second plurality ofdiffractive light beams has a different phase. The first plurality ofdiffractive light beams are therefore spatially distinct from oneanother, and the second plurality of diffractive light beams are alsospatially distinct from one another. Additionally, each of the firstdiffractive light beams includes light from a different depth planewithin the sample, and each of the second diffractive light beamsincludes light a different depth plane within the sample. The firstdiffractive element can be positioned in a Fourier plane of a firstoptical arm of the system, and the second diffractive element can bepositioned in a Fourier plane of a second optical arm of the system.

In some embodiments, the multi-focus gratings focus the first pluralityof diffractive light beams and the second plurality of diffractive lightbeams by applying a phase shift that is equal to but opposite from adepth-induced phase error that is present on the light that emerges fromout-of-focus planes in the sample, As discussed above, these phaseshifts can be applied as a result of the geometrical distortion in thegrating pattern of the multi-focus grating. In some embodiments, themulti-focus gratings apply the phase shift to a first wavefront of thelight of the first plurality of diffractive light beams and to a secondwavefront of the light of the second plurality of diffractive lightbeams, where the first wavefront and the second wavefront have conjugatephase.

In an operation 320, the system simultaneously directs and focuses eachof the first plurality of diffractive light beams and each of the secondplurality of light beams onto different portions of an image plane of adetector. The detector can be an EMCCD detector as described herein. Inan operation 325, based on the data received by the detector (i.e., thefirst and second plurality of diffracive light beams), the systemgenerates one or more 3D image representations of the sample. The 3Dimage representations can each be compilations of 2D images representingthe various depth planes within the sample.

Described below is a mathematical process to derive the iMFM pointspread function (PSF) by assuming uniform excitation light at thesample. The derivation is an extension of single-objective MFM to thedual-objective iMFM. As an example, a single point source (0, 0, z) maybe sandwiched between two opposing objectives, where z is thedisplacement of the point from the sample focal plane. Upon fluorescenceemission, the first and second objective lenses can respectively producecomplex wavefronts on the detector plane (x, y) as follows:

E _(1,iMFM)(x, y; 0, 0, z; λ)=

{g ₁(x _(g) , y _(g); λ)f ₁(x _(g) , y _(g); 0, 0,z; λ)},  Eq. 1:

E _(2,iMFM)(x, y; 0, 0, z; λ)=

{g ₂(x _(g) , y _(g); λ)f ₂(x _(g) , y _(g); 0, 0,z; λ)},  Eq. 2:

where (x, y) are spatial coordinates on the detector plane, λ is thefluorescence emission wavelength,

denotes the 2D Fourier transform, g₁ and g₂ denote MFGs placed in theFourier planes of the two objective lenses, respectively, with spatialcoordinates of (x_(g), y_(g)), and f₁ and f₂ are complex wavefronts inthe first and second objective Fourier plane caused by the point source(0, 0, z) propagation, respectively. Continuing the example, it can beshown that

{f₁(x_(g), y_(g); 0, 0, z; λ)}=p₁(x, y; 0, 0, z; λ) and

{f₂(x_(g), y_(g); 0, 0, z; λ)}=p₂(x, y; 0, 0, z; λ), where p₁(x, y; 0,0, z; λ) is the coherent spread function (CSF) of a point source (0, 0,z) in a single lens imaging under uniform illumination. This has thefollowing form in the focal region of a high numerical aperture (NA)objective of circular aperture under the scalar and Debyeapproximations:

$\begin{matrix}{{{p_{1}\left( {x,{y;0},0,{z;\lambda}} \right)} = {\frac{A}{\lambda}{\int_{0}^{\alpha}{\sin\;\theta\sqrt{\cos\;\theta}{\exp\left( {{- {ikz}}\;\cos\;\theta} \right)}{J_{0}\left( {k\;{\rho\left( {x,y} \right)}\sin\;\theta} \right)}d\;\theta}}}},} & {{Eq}.\mspace{14mu} 3}\end{matrix}$

where A is a constant, α=sin⁻¹(NA/n₀)) is the semi-aperture angle of theobjective lens, in which n₀ is the index of refraction, k=2πn₀/λ is thewave number, J₀ is the zeroth order Bessel function of the first kind,and ρ(x, y)=√{square root over (x²+y²)} denotes the radial coordinateposition on the detector plane. It is noted that √{square root over (cosθ)} is an apodization function for a high NA objective under Abbe sinecondition. The CSF modeling with a low NA objective can be derived fromEq. 3 by further assuming the paraxial approximation. In dual-objectivedetection, there exists the following relationship:

p ₂(x, y; 0, 0, z; λ)=p ₁(x, y; 0, 0, −z; λ)  Eq. 4:

because of opposite propagation directions of the emitted light intoboth the first and second objectives.

For monochromatic light, the detection intensity PSF is the square ofthe coherent addition of spherical wavefronts of two opposing objectivelenses, expressed as:

h _(iMFM) ^(mono)(x, y; 0, 0, z; λ)=|E _(1,iMFM)(x, y; 0, 0, z; λ)+E_(2,iMFM)(x, y; 0, 0, z; λ)|².  Eq. 5:

As expected, if no MFGs are placed in the Fourier planes of the twoobjective lenses (which means g₁ and g₂ are all inside the circularpupil plane), Eq. 5 reduces to the detection PSF of I²M or I⁵M, whichrecords interferometric information from one focal plane at a time, andthen assembles a 3D interferometric stack from sequentially refocused 2Dimages.

In the iMFM configuration shown in FIG. 2A, the entire 3Dinterferometric focal stack can be recorded in a single shot by placingtwo MEGs of conjugate focal shifts in the Fourier planes of dualobjectives. This is because the MFG can be designed to split the emittedbeam into an array of (2M+1)×(2N+1) differently focused tiles on thedetector at one exposure time, where M and N are design parameters ofthe system which determine the total number of the tiles. The focalshift property of the MFG is achieved by imposing a geometricaldistortion on a regular periodic grating pattern (with periods d_(x) andd_(y) in the x and y directions, respectively) placed in the Fourierplane. This geometrical distortion in return introduces anorder-dependent defocus phase shift in the detector plane.Mathematically, the MFG equation can be described as (see Appendix formore details):

$\begin{matrix}{{{\mathcal{F}\left\{ {{\mathcal{g}}_{1}\left( {x_{\mathcal{g}},{y_{\mathcal{g}};\lambda}} \right)} \right\}} = {\sum_{m = {- M}}^{M}{\sum_{n = {- N}}^{N}{{w_{m,n}(\lambda)}{\exp\left( {{- {ikz}_{m,n}}\frac{\lambda}{\lambda_{c}}\cos\;\theta} \right)}{\delta\left( {{x - {{mx}_{0}\frac{\lambda}{\lambda_{c}}}},{y - {{ny}_{0}\frac{\lambda}{\lambda_{c}}}}} \right)}}}}},} & {{Eq}.\mspace{14mu} 6} \\{{{\mathcal{F}\left\{ {{\mathcal{g}}_{2}\left( {x_{\mathcal{g}},{y_{\mathcal{g}};\lambda}} \right)} \right\}} = {\sum_{m = {- M}}^{M}{\sum_{n = {- N}}^{N}{{w_{m,n}(\lambda)}{\exp\left( {{ikz}_{m,n}\frac{\lambda}{\lambda_{c}}\cos\;\theta} \right)}{\delta\left( {{x - {{mx}_{0}\frac{\lambda}{\lambda_{c}}}},{y - {{ny}_{0}\frac{\lambda}{\lambda_{c}}}}} \right)}}}}},} & {{Eq}.\mspace{14mu} 7}\end{matrix}$

where w_(m,n) ² is the diffraction efficiency of diffraction order (m,n) such that Σ_(m=−M) ^(M)Σ_(n=−N) ^(N)w_(m,n) ²≤1 is the totalefficiency of MFG, z_(m,n)=(m+Bn)Δz is a focal shift at diffractionorder (m, n), in which Δz is a predefined focal step between twoadjacent focal planes and B=2M+1, λ_(c) is the central emissionwavelength used for the MFG distortion calculation, δ(x, y) is a Diracdelta function, and (mx₀, ny₀)=(mf₂λ_(c)/d_(x), nf₂λ_(c)/d_(y)) is thecenter position of diffractive order (m, n) on the camera plane underthe paraxial approximation for the emission wavelength λ_(c). It isnoted that the paraxial approximation holds here because the focallength f₂ of the relay system lens is much larger than the sensor size.The above equations indicate two distinct properties of the MFG, namelylight path splitting into an array of (2M+1)×(2N+1) diffraction orders(or tiles) indicated by the Dirac delta function due to the periodicproperty of the MFG, and an order-dependent phase shift indicated by theexponential phase function due to the distortion of the MFG.

If the wavefront defocus phase from the out-of-focus plane (exp(ikz cosθ) as shown in Eq. 3) is compensated (or corrected) by the MFGorder-dependent defocus phase (exp(−ikz_(m,n) cos θλ/λ_(c)), as shown inEq. 3), it is possible to simultaneously focus the light originatingfrom an in-focus plane and multiple out-of-focus planes onto distinctlateral diffraction orders, and therefore form multi-focus images on asingle 2D camera within one exposure time. This is one of the principlesof a single lens MFM system. In the dual-objective iMFM configuration,the light originating from the same emitted point source has conjugatephase after passing through two opposing objectives. Therefore, in orderto diffract these two conjugate spherical wavefronts to the samediffraction order on the camera, two MFG g₁ and g₂ with conjugate focalshifts are placed in the Fourier planes in the iMEM microscope, as shownin FIG. 2A. In an alternative embodiment, the use of two phase masks toproduce the correct interference between two coherent beams of light canbe used.

According to the convolution theorem, by substituting Eq. (3-4, 6-7),Eqs. 1 and 2 become:

$\begin{matrix}{{\left. {E_{1,{iMFM}}\left( {x,{y;0},0,{z;\lambda}} \right)} \right) = {\sum_{m = {- M}}^{M}{\sum_{n = {- N}}^{N}{{w_{m,n}(\lambda)}{p_{1}\left( {{x - {{mx}_{0}\frac{\lambda}{\lambda_{c}}}},{{y - {{ny}_{0}\frac{\lambda}{\lambda_{c}}}};0},0,{z - {z_{m,n}\frac{\lambda}{\lambda_{c}}}}} \right)}}}}},} & {{Eq}.\mspace{14mu} 8} \\{{\left. {E_{2,{iMFM}}\left( {x,{y;0},0,{z;\lambda}} \right)} \right) = {\sum_{m = {- M}}^{M}{\sum_{n = {- N}}^{N}{{w_{m,n}(\lambda)}{p_{1}\left( {{x - {{mx}_{0}\frac{\lambda}{\lambda_{c}}}},{{y - {{ny}_{0}\frac{\lambda}{\lambda_{c}}}};0},0,{{z_{m,n}\frac{\lambda}{\lambda_{c}}} - z}} \right)}}}}},} & {{Eq}.\mspace{14mu} 9}\end{matrix}$

and Eq. (5) becomes

$\begin{matrix}{{{h_{iMFM}^{mono}\left( {x,{y;0},0,{z;\lambda}} \right)} = {\sum_{m = {- M}}^{M}{\sum_{n = {- N}}^{N}{h_{m,n}\left( {x,{y;0},0,{z - {z_{m,n}\frac{\lambda}{\lambda_{c}}}}} \right)}}}},{where}} & {{Eq}.\mspace{14mu} 10} \\{{h_{m,n}\left( {x,{y;0},0,{z - {z_{m,n}\frac{\lambda}{\lambda_{c}}}}} \right)} = {\quad{\quad\left. {w_{m,n}^{2}(\lambda)} \middle| {\frac{\hat{M}}{f_{obj}^{2}\lambda^{2}}{\int_{0}^{\alpha}\left\{ {{\exp\left\lbrack {{{ik}\left( {z - {z_{m,n}\frac{\lambda}{\lambda_{c}}}} \right)}\cos\;\theta} \right\rbrack} + {\quad{\left. \quad{\exp\left\lbrack {{{ik}\left( {{z_{m,n}\frac{\lambda}{\lambda_{c}}} - z} \right)}\cos\;\theta} \right\rbrack} \right\} \times {\quad{{\sin\;\theta\sqrt{\cos\;\theta}J_{0}{\quad{\left\lbrack {{\rho\left( {{x - {{mx}_{0}\frac{\lambda}{\lambda_{c}}}},{y - {{ny}_{0}\frac{\lambda}{\lambda_{c}}}}} \right)}\sin\;\theta} \right\rbrack d\;\theta}}^{2}},}}}}} \right.}} \right.}}} & {{Eq}.\mspace{14mu} 11}\end{matrix}$

is a tile-PSF at diffraction order (m, n).

The polychromatic intensity PSF is an integration of the monochromaticPSF over the emission spectrum Δλ:

$\begin{matrix}{{h_{iMFM}^{poly}\left( {x,{y;0},0,{z;\lambda}} \right)} = {\sum_{m = {- M}}^{M}{\sum_{n = {- N}}^{N}{\int_{\lambda_{c} - {{\Delta\lambda}/2}}^{\lambda_{c} + {{\Delta\lambda}/2}}{{h_{m,n}\left( {x,{y;0},0,{z - {z_{m,n}\frac{\lambda}{\lambda_{c}}}}} \right)}d\;{\lambda.}}}}}} & {{Eq}.\mspace{14mu} 12}\end{matrix}$

Equations 10 and 12 are analytical formulations of the monochromatic andpolychromatic PSFs of the dual-objective iMFM. They indicate that bothmonochromatic and polychromatic PSFs consist of multiple tile-PSFs. Eachtile-PSF has a distinct focal plane of z_(m,n)λ/λ_(c) for emissionwavelength λ, and more importantly, features an axial interferencepattern, which contains high frequencies and high resolutioninformation.

To verify the iMFM imaging system and make a practical comparisonbetween MFM and iMFM PSFs, simulations wee conducted for the detectionPSF for a numerical aperture NA=1.27 of water immersion objective, withrefractive index of 1.338, magnification {circumflex over (M)}=60, andthe emission central wavelength λ_(c)=620 nm. Based on these parameters,a first multi-focus grating (MFG1) was designed, which produces 3×3focal shift images with a focal step of Δz=250 nm. FIG. 4A depicts thefirst multi-focus grating with focal step Δz=250 nm in accordance withan illustrative embodiment. The period of the first multi-focus gratingis d_(x)=d_(y)=56 μm with a pixel size of 1 μm, and the diameter of MFG1is set to be the same as that of the pupil aperture, defined by2f_(obj)NA=8467 μm. The unit cell pattern of MFG1 is optimized for amaximum diffraction efficiency by using the iterative Gerchberg-Saxton(GS) algorithm. As a result, the diffraction efficiency w_(m,n) ² foreach diffraction order is [7.13, 7.79, 7.01, 7.70, 8.57, 7.70, 7.01,7.79, 7.13] percent with a total efficiency of 67.83%, which is close tothe theoretical maximum efficiency. The geometric distortion is thenimposed on MFGI to create a proper focal step of Δz=250 nm betweenconsecutive diffraction orders for the emission central wavelengthλ_(c). A second multi-focus grating (MFG2) with opposite focal shift wasgenerated by rotating MFG1 by 180 degrees. FIG. 4B depicts the secondmulti-focus grating with a focal step of Δz=250 nm in accordance with anillustrative embodiment. In FIGS. 4A and 4B, the insets depict thecentral 400×400 pixels of the respective MFGs, which contain multiplegrating unit cells.

Using the system with the designed MFGs (MFG1 and MFG2) depicted in FIG.4, the detection PSFs were numerically simulated in the size of1500×1500×200 voxels with the voxel size of 80 nm×80 nm×1.0 nm for bothMFM and iNIFM. The monochromatic unaberrated iMFM detection PSF iscomputed from Eq. 5 by setting λ=λ_(c). For a polychromatic PSF in thepresence of chromatic aberrations (CA), a 10 nm emission filter isconsidered and the PSF is computed by integrating Eq. 5 over theemission bandwidth of 10 nm. FIG. 5 depicts the xy, xz, yz cuts and 1Daxial profile of monochromatic and polychromatic MFM and iMFM PSFs inaccordance with an illustrative embodiment. Specifically, FIG. 5Adepicts an xy cut, an xz cut, a yz cut, and an axial profile of eachtile PSF for an MFM monochramatic PSF in accordance with an illustrativeembodiment. FIG. 5B depicts an xy cut, an xz cut, a yz cut, and an axialprofile of each tile PSF for an iMFM monochramatic PSF in accordancewith an illustrative embodiment. FIG. 5C depicts an xy cut, an xz cut, ayz cut, and an axial profile of each tile PSF for an MFM polychramaticPSF in accordance with an illustrative embodiment. FIG. 5D depicts an xycut, an xz cut, a yz cut, and an axial profile of each tile PSF for aniMFM polychramatic PSF in accordance with an illustrative embodiment.

FIG. 5 verifies that both monochromatic and polychromatic iMFM FM PSFsare composed of multiple focal shift interferometric tile-PSFs, asindicated by Eqs. 10 and 12, respectively. When CA exits, eachnon-central tile-PSF has a dispersion along the x and/or y directions,causing the peak intensity to become relatively lower, as shown in the1D axial profile of FIGS. 5C and 5D. However, full width at half maximum(FWHM) along the z direction remains almost the same, with about 190 nmfor iNIFM PSF and 620 nm for MFM PSF, with a 3.3-fold decrease of thediffraction spot size along the axial direction.

The proposed dual-objective iMFM features a single-shot multifocalinterferometry detection, mapping multiple interferotnetric focal planesof a 3D sample volume o(x, y, z) simultaneously onto a 2D image planeI(x, y) in one exposure time without translating the sample. In thiscase, the recorded intensity is given by

I(x, y)=

{∫_(z) o(x, y; z)*h _(iMFM)(x, y; z)dz+b}+{circumflex over (n)},  Eq.13:

where

represents Poisson statistics originating from signal photons, * denotesthe 2D convolution, i_(iMFM) is the iMFM monochromatic or polychromaticPSF, b is homogeneous background noise, and {circumflex over (n)}denotes additive Gaussian noise. For compaction, Equation 13 can also bewritten in a matrix-vector form as

I=

{Ao+b},  Eq. 14:

where I is an

+1 column vector, in which

=

_(x)

_(y) is the number of pixels of the recorded image, and o is an

×1 column vector, in which

=

_(x)

_(y)

_(z) is the number of voxels of the 3D unknown object to be recovered,and A is the sensing matrix of a size

×

, computed from iMFM's normalized PSF h_(iMFM), whose integral is equalto one. The additive Gaussian noise is ignored here for two reasons.First, MFM and iMFM are photon budget limited detection schemes becausethe emitted photons are split into multiple light paths in order tocreate proper refocus without overlapping, and thus, the Poisson noisedominates. Second, the read-out noise of an electron multiplying chargecoupled device (EMCCD), which is commonly used in low light conditionfluorescent imaging, is neglectable due to the high electron multiplying(EM) factor.

In order to take the Poisson noise into account and remove theout-of-focus blur, the Richardson-Lucy (R-L) deconvolution algorithm foriMFM 3D extended object recovery was used. To suppress noiseamplification, total variation (TV) regularization is used. In addition,the non-negativity constraint is applied due to the non-negative natureof fluorescent light so as to restrict the set of possible solutions.These constraints are helpful when more 3D information from fewer 2Dmeasurement data in MFM iss recovered, i.e., when

<

. In R-L deconvolution, the following optimization is performed:

$\begin{matrix}\begin{matrix}{{\underset{o}{\arg\;\min}{\sum_{p}\left\{ {{{- {I(p)}}{\log\left\lbrack {\left( {{Ao} + b} \right)(p)} \right\rbrack}} + {\left( {{Ao} + b} \right)(p)}} \right\}}} + {\lambda_{TV}{{TV}(o)}}} \\{{{subjecttoo} \geq 0},}\end{matrix} & {{Eq}.\mspace{14mu} 15}\end{matrix}$

where p denotes the pixel coordinate in the captured image, λ_(TV) isthe regularization parameter, and TV(o)=Σ_(s)|∇o(s)|, in which s denotesthe voxel coordinate in the o. A solution to the optimization problem inEquation 15 can be found by the following iteration:

$\begin{matrix}{{{o_{k + 1}(s)} = {\left\{ {\left\lbrack {A^{t}\left( \frac{I}{{Ao}_{k} + b} \right)} \right\rbrack(s)} \right\}\frac{o_{k}(s)}{1 - {\lambda_{TV}{{div}\left( \frac{\nabla{o_{k}(s)}}{{\nabla{o_{k}(s)}}} \right)}}}}},\mspace{14mu}{{o_{k + 1}(s)} \geq 0},} & {{Eq}.\mspace{14mu} 16}\end{matrix}$

where k denotes the iteration number, the divisions are element wise,A^(t) is the transpose of A, and div stands for the divergence operator.The denominator of Equation 16 may become zero or negative due to alarge value of λ_(TV). To prevent this from happening, the negativevalues or ‘not a number’ (NAN) values were set to zero at each iterationstep. The algorithm terminates, when the difference between twoconsecutive values of the cost function is smaller than a predefinedthreshold.

In single particle tracking, the 3D space includes a single point withvarying 3D positions over time. The single point can be modeled aso=δ(θ−{circumflex over (θ)})=δ(x−{circumflex over (x)}₀, y−ŷ₀,z−{circumflex over (z)}₀) and maximum likelihood estimation (MLE) can beused to recover its 3D position {circumflex over (θ)}=({circumflex over(x)}₀, ŷ₀, {circumflex over (z)}₀). In MLE, the following optimizationis performed:

$\begin{matrix}{\underset{\theta}{argmin}{\sum_{p}{\left\{ {{{- {I(p)}}{\log\left\lbrack {\left( {{h_{iMFM}(\theta)} + b} \right)(p)} \right\rbrack}} + {\left( {{h_{iMFM}(\theta)} + b} \right)(p)}} \right\}.}}} & {{Eq}.\mspace{14mu} 17}\end{matrix}$

Equation was minimized using the interior-point method of Matlab fminconfunction. Alternatively, a different minimization algorithm may be used.It should be noted that the optimization problem in Equation 17 for MLElocalization is non-convex, and therefore it is sensitive to the initialpoint. To improve the accuracy of MLE, a method is described below todetermine the initial axial position of the single point that is imagedby the iMFM microscope.

A Fisher information matrix (FIM) measures the sensitivity of anobservation (e.g., iMFM PSF) to changes of the parameters to beestimated (e.g., 3D position of a single molecule). The model forcalculating the FIM for iMFM is the same as that for MFM. For each tileimage, the photon detection is an independent Poisson process.Therefore, the total FIM of an iMFM PSF is the sum of the FIM for eachtile PSF and can be expressed as a 3×3 matrix, as follows:

$\begin{matrix}{{F = \begin{bmatrix}{F_{xx},} & {F_{xy},} & F_{xz} \\{F_{yx},} & {F_{yy},} & F_{yz} \\{F_{zx},} & {F_{zy},} & F_{zz}\end{bmatrix}},} & {{Eq}.\mspace{14mu} 18}\end{matrix}$

where each entry of the matrix is

$\begin{matrix}{{F_{ij} = {\sum_{m = {- M}}^{M}{\sum_{n = {- N}}^{N}{\sum_{p = 1}^{N_{p}}{\frac{N_{m,n}^{2}}{{N_{m,n}{{\hat{h}}_{m,n}(p)}} + b}\frac{\partial{{\hat{h}}_{m,n}(p)}}{\partial i}\frac{\partial{{\hat{h}}_{m,n}(p)}}{\partial j}}}}}},} & {{Eq}.\mspace{14mu} 19}\end{matrix}$

in which i,j∈[x, y, z], N_(p) is the number of pixels for each tileimage, b is the homogeneous background photons per pixel, ĥ_(m,n)(p) isa normalized tile-PSF at diffraction order (m, n), and M_(m,n)=w_(m,n)²2N_(tot)γ denotes the number of photons collected by the tile-PSF,where N_(tot) denotes the total number of photons collected by eachobjective lens and γ=0.5 denotes the photon loss ratio at the beamsplitter (BS). Then the Cramer-Rao lower bound (CRLB), which bounds thevariance of the localization estimation σ², can be calculated by takingthe diagonal elements of the inverse of the FIM as:

$\begin{matrix}{\begin{bmatrix}{CRLB}_{x} \\{CRLB}_{y} \\{CRLB}_{z}\end{bmatrix} = {\begin{bmatrix}\sigma_{x}^{2} \\\sigma_{y}^{2} \\\sigma_{z}^{2}\end{bmatrix} = {{{Diag}\left( F^{- 1} \right)}.}}} & {{Eq}.\mspace{14mu} 20}\end{matrix}$

It can be seen from Equation 19 that the Fisher information or thelocalization precision can be improved by increasing the derivative ofthe PSF, i.e., ∂ĥ_(m,n)/∂i. In the dual-objective iMFM, each tile-PSFhas a higher z-derivative due to its axial features of in theinterference pattern, and therefore a higher axial differentialinformation content, causing 3 to 4-fold improvement along axiallocalization compared with single lens MFM. In addition, thesimultaneous multifocal detection of iMFM leads to almost uniformly highcombined differential information content along the large depth range,while conventional dual-objective detection in a single channel suffersfrom non-uniform localization along z due to zero z-derivative at itsPSF intensity nodes. FIG. 6 shows xz-cuts of the square of z-derivativeof the normalized MFM and iMFM PSFs. Specifically, FIG. 6A depicts xzcuts of the square of the z-derivative for MFM monochromatic PSF (left),MFM polychromatic PSF (middle left), iMFM monochromatic PSF (middleright), and iMFM polychromatic PSF (right) in accordance with anillustrative embodiment. As shown, the z-derivative is higher for thedual objective iMFM detection due to the steepended axial features ofinterferometric iMFM PSF. FIG. 6B depicts xz cuts of the combinedz-derivative summed over 3×3 tiles in FIG. 6A in accordacne with anillustrative embodiment. As shown, the combination of 9 tiles leads toalmost uniformly high infomration content along the optical axis.

According to Equation 19, the resolution can also be improved byincreasing the number of the collected photons, i.e., N_(m,n). Thedual-objective detection collects twice the number of photons comparedwith single lens imaging, and therefore improves the resolution by afactor of √{square root over (2)} in all three dimensions when thebackground photon b is small. However, the light efficiency of thedual-objective iMFM configuration shown in FIG. 2A is similar to that ofsingle objective MFM in that there is a factor of two light loss in theBS. To avoid the light loss, a second camera can be introduced andplaced at the output of the BS. The axial resolution improvement isdemonstrated in FIG. 7A, which is a plot of the theoretical localizationprecision σ_(z) along a 2 μm axial range calculated from Equations 19and 20. More specifically, FIG. 7A depicts theoretical axiallocalization precision σ_(z) for abberated and unaberrated detectionschemes using the proposed iMFM and abberated and unaberrated detectionschemes using an MFM system in accordance with an illustrativeembodiment. For the calculation, 2500 detected signal photons perobjective lens and 10 background photons per pixel were considered. FIG.7B depicts the mean squared error of the z position determined during 50simulated localizations per individual axial value for MFM and iMFM andcorresponding theoretical predictions in accordance with an illustrativeembodiment.

In the calculation, N_(tot)=2500 detected signal photons per objectivelens and b=10 background photons per pixel werere considered, which aretypical values observed in single molecule experiments. The parametervalues of the microscope and MFG are set to be the same as those whenthe PSF was simulated, as described above. The results suggest that themonochromatic iMFM PSF provides an average theoretical localizationprecision of (σ_(x), σ_(y), σ_(z))=(16.6 nm, 16.6 nm, 11.2 nm) over theimaging range of 2 μm for 2500 signal photons and 10 background photons.Compared to the single lens monochromatic MFM PSF with localizationprecision of (18.0 nm ,18.0 nm, 44.8 nm), the lateral localizationprecision √{square root over (σ_(x)σ_(y))} is increased by a factor of0.08 (this small gain results from the redistribution of the light dueto the interference), and the axial one by a factor of 4. When chromaticaberration exists in the presence of a 10 nm emission spectrum, both MFMand iMFM localization precisions decrease. An iMFM polychromatic PSF canachieve average localization precision of (σ_(x), σ_(y), σ_(z))=(25.6nm, 24.0 nm, 16.6 nm), with 1.17-fold improvement laterally and 3.6-foldimprovement axially compared to MFM polychromatic PSF of localizationprecision (29.9 nm, 28.3 nm, 59.7 nm). The localization precision isslightly different in the x and y directions for polychromatic aberratedPSFs. This is because the light diffraction efficiency is different forhorizontal and vertical diffraction orders of the designed MFG. Thoseresults indicate that iNIFM could provide 3 to 4-fold improved axiallocalization precision in both aberrated and unaberrated systems.

In order to verify the theoretical analysis discussed above anddemonstrate the capability of iMFM to achieve higher axial localizationprecision, MLE reconstructions of single particle tracking using iMFMmultifocal interferometric detection were performed. However, it isknown that the optimization problem in MLE as shown in Equation isnon-convex, and a global minimal is not guaranteed to be found.Therefore, multiple random initial values are used for the MLElocalization algorithm, and the optimal solution with the minimal costfunction values is picked as the final reconstruction.

A better initial value closer to the global minimal than random initialvalues could improve the success and convergence of the MLE localizationalgorithm, but it is difficult to determine since there is no priorinformation about the 3D position of the point in conventionalmicroscopy imaging. In addition, it is impossible to tell whether thepoint is in the positive or negative defocus because the PSF issymmetric with respect to the focal plane and has the same blur size forequal magnitude but opposite defocus.

Described below is a method to determine an initial axial position ofthe single point that is imaged by the MFM and iMFM microscopes. In MFMand iMFM, the PSF is not symmetric any more due to the multi-focusingproperty. Furthermore, each z position point is focused in differenttiles, expressed as z=(m+Bn)Δz, where (m, n) is the focused tilediffraction order, and B and Δz are the pre-designed parameters whichare known a priori. Therefore, if one can determine which tile image ismost in focus by comparing the blur sizes of the tile-PSFs, then theinitial axial position of the molecule z₀ can be found as z₀=(m+Bn)Δz.Also, since the focal step between two consecutive tiles is Δz, theerror distance between the initial estimation and the ground truthshould be smaller than Δz, i.e., |z₀−{circumflex over (z)}|≤Δz. In thesimulation, |z₀−{circumflex over (z)}|≤2Δz was used in order to preserveboth accuracy and speed of the MLE localization algorithm.

FIG. 8 depicts an example of determining the initial axial position andrange of a single point for an MLE localization algorithm in iMFM inaccordance with an illustrative embodiment. In the simulation, Ntot=2500total detected signal photons and b=10 background photons per pixel wereconsidered. In FIG. 8, the left tile depicts the focal shift of the MFGas known a priori, the right tile depicts the simulated iNIFM image of asingle point, and the relationships on the right are the initial axialposition and range estimations.

It is known that for each position ({circumflex over (x)}₀, ŷ₀,{circumflex over (z)}₀) of the molecule emitter, the acquired pixelatedimage under Poisson noise corruption is generated as

I(x, y; {circumflex over (x)} ₀ , ŷ ₀ , {circumflex over (z)} ₀)=

[N _(tot)∫∫_(C) _(p) ĥ _(iMFM)(x−{circumflex over (x)} ₀ , y−ŷ ₀ ;{circumflex over (z)} ₀)dxdy+b],  Eq. 21:

where C_(p) denotes the pixel area on the detector plane, ĥ_(iMFM) isthe normalized iMFM PSF with its integral equal to one, and N_(tot) isthe total number of the photons collected by the iMFM PSF. Themicroscope and MFG parameters were the same as those used for PSFsimulation and CRLB calculation described above. The acquired image Iwas 3×3 tile images with a focal shift of Δz=0.25 μm. It was assumedthat each tile image had a region of interest (ROI) of 41×41 pixels,with a pixel size of 4 μm×4 μm. In addition, each pixel was composed of4×4 sub-pixels for the purpose of the integral over the pixel areaC_(p).

Starting with a proper initial value as described above, 50 images foreach z position of the emitter between −1 μm and 1 μm from Equation 21were simulated and MLE was used to back-calculate the 3D position of theemitter. For each 3D position, a cluster of the positions containing 50points was recovered and the mean squared error between estimatedposition and true position was calculated. The estimation errors in theaxial dimension by both MFM and iMFM are shown in FIG. 7B. Thesimulation results indicate that the MLE estimation errors are wellconsistent with theoretical predictions. A trajectory of a singleemitter which follows a random walk from −1 μm toward 1 μm were alsosimulated, and plotted in 3D view along with its projections onto thexy, xz and yz planes. The MFM and iMFM reconstructed trajectories werealso plotted. The difference between the ground truth andreconstructions along the axial direction is also plotted as a histogramof axial residuals. FIG. 9A depicts a histogram of axial residualsbetween ground truth and MFM recovery in accordance with an illustrativeembodiment. FIG. 9B depicts a historgram of axial residuals betweenground truth and iMFM recovery in accordance with an illustrativeembodiment. The standard deviation for MFM is 48.37 nm and standarddeviation for iMFM is 12.12 nm, resulting in a 4-fold boost in axiallocalization precision that iMFM detection PSF can offer over a largevolume.

Described below is isotropica 3D resolution for iMFM wide field imaging.For a band-limited system, the Nyquist sampling rate has to be satisfiedin order to avoid aliasing. In the dual-objective iMFM microscope, thelateral cut-off frequency is 2NA/λ and the axial cut-off frequency is2{circumflex over (n)}/λ. Therefore, the Nyquist sampling distance inthe sample space has to be equal or less than Δ_(xy)=λ/(4NA) in thelateral direction and Δ_(z)=λ/(4{circumflex over (n)}) in the axialdirection.

For an iMFM system with NA=1.27, index of the refraction {circumflexover (n)}=1.338 and λ=620 nm, it was found that Δ_(xy)=122 nm andΔ_(z)=116 nm. In the microscope design, the lateral sampling distancecan be met by choosing a camera with proper pixel size such thatd_(pixel)/{circumflex over (M)}≤Δ_(xy), and the axial sampling distancecan be met by designing the MFG with a focal step |Δz|≤Δ_(z).

To confirm that iMFM provides axial super-resolution and hence 3Disotropic resolution in the wide field fluorescence imaging, imaging ofa 3D synthetic extended object was also performed. The same parametervalues for the microscope were used: NA=1.27 with index of refraction1.338, and magnification {circumflex over (M)}=100. In order to recordmore than 9 focal images in a single shot, a new MFG that produces anarray of 5×5 focal shift images was designed. The focal step wasdesigned to be Δz=100 nm in order to satisfy the Nyquist-Shannonsampling condition along z. The sensor size is assumed to be 1024×1024pixels, with pixel size of 12 μm×12 μm. The size of each focal shifttile image is about 205×205 pixels. To avoid lateral convolutionartifacts at the boundary, the lateral field of view (FOV) of a 3Dsynthetic extended object is confined to a central part of 129×129pixels. The 3D synthetic object resembled the structure of microtubes,and is of size 129×129×49 voxels where each voxel size is 120 nm×120nm×50 nm.

The single shot 2D measurements for iMFM and MFM were simulated usingEquation 13. To simulate a microscope with chromatic aberrations (CA),the emission bandwidth of 10 nm was considered by using a 10 nm filter.The maximum number of photons detected by the brightest pixel was 500,and the corresponding Poisson noise was added in each measurement. Forthe reconstruction, the Richardson-Lucy algorithm with total variation(TV) regularization was used, as discussed above. The optimalregularization parameters were found by exhaustive search and thealgorithm was run to converge, when the difference between twoconsecutive values of the cost function is smaller than a predefinedthreshold.

FIG. 10A depicts the 3D extended test object in accordance with anillustrative embodiment. FIG. 10B depicts recovery of the 3D extendedtest object using MFM in accordance with an illustrative embodiment.FIG. 10C depicts recovery of the 3D extended test object usingunaberrated iMFM in accordance with an illustrative embodiment. FIG. 10Ddepicts recovery of the 3D extended test object using aberrated iMFM inthe presence of CA in accordance with an illustrative embodiment. Ineach of FIGS. 10A-10D, the first row depicts a 3D image in an xy slice,the second row depicts a first xz slice, the third row depicts a secondxz slice, the fourth row depicts a comparison of linecuts and spectrabetween the ground truth, MFM, and iMFM reconstructions, and the bottomrow depics the comparisin of k_(z)k_(x) spectra by Fouriertransformation of the reconstructions. The depicted results demonstratethat both aberrated and unaberrated iMFM can receover higher axialspatial frequencies beyond the detection cut-off of the single lens MFMsystem, and can therefore achieve super-resolution in the axialdirection even for a system with chromatic aberrations. As shown in FIG.10B, two axial finer features separated by approximately 375 nm axiallyare blurred to a single feature in the MFM reconstruction. However,these features are well resolved by both unaberrated and aberrated iMFMreconstructions as shown in FIGS. 10C and 10D.

Included below are the derivation details of the multifocus grating(MEG) equation (i.e., Equation 6). A normal periodic grating g withspacing d_(x) and d_(y) in the x and y directions, respectively, wasconsidered. Because the grating is periodic and continuous, the Fouriertransform (FT) of it yields a discrete and aperiodic spectrum asfollows:

{g(x _(g) , y _(g); λ)}=Σ_(m=−M) ^(M)Σ_(n=−N) ^(N) w _(m,n)(λ)δ(u−mu ₀ ,v−nv ₀),  Eq. 22:

where u and v are spatial frequencies in the x and y directions,respectively, and u₀=1/d_(x) and v₀=1/d_(y) are the intervals betweenconsecutive samples in the discrete spectrum of the grating, In Fourieroptics, u=x/(fλ) and v=y/(fλ) under the paraxial approximation, where xand y are spatial coordinates in the detection plane. It is noted thatthe paraxial approximation holds here because the focal length f of therelay system lens is much larger than the sensor size.

For MFG, the geometrical distortions Δ_(x) and Δ_(y) are introduced inthe grating pattern in the x and y directions, respectively. Therefore,according to the FT shift theorem, the FT of the distorted grating g₁can be written as

$\begin{matrix}{{\mathcal{F}\left\{ {{\mathcal{g}}_{1}\left( {x_{\mathcal{g}},{y_{\mathcal{g}};\lambda}} \right)} \right\}} = {{\mathcal{F}\left\{ {{\mathcal{g}}\left( {{x_{\mathcal{g}} - \Delta_{x}},{{y_{\mathcal{g}} - \Delta_{y}};\lambda}} \right)} \right\}} = {\quad{\sum_{m = {- M}}^{M}{\sum_{n = {- N}}^{N}{{w_{m,n}(\lambda)}{\exp\left\lbrack {{- i}\; 2{\pi\left( {{{mu}_{0}\Delta_{x}} + {{nv}_{0}\Delta_{y}}} \right)}} \right\rbrack}{{\delta\left( {{u - {mu}_{0}},{v - {nv}_{0}}} \right)}.}}}}}}} & {{Eq}.\mspace{14mu} 23}\end{matrix}$

In Equation 23, the geometrical distortions are set to beΔ_(x)=d_(x)n₀Δz cos θ/λ_(c) and Δ_(y)=Bd_(y)n₀Δz cos θ/λ_(c) to create aproper refocus. Therefore, Equation 23 can be rewritten as

$\begin{matrix}{{\mathcal{F}\left\{ {{\mathcal{g}}_{1}\left( {x_{\mathcal{g}},{y_{\mathcal{g}};\lambda}} \right)} \right\}} = {\quad{{\sum_{m = {- M}}^{M}{\sum_{n = {- N}}^{N}{{w_{m,n}(\lambda)}{\exp\left( {{- i}\;{kz}_{m,n}\frac{\lambda}{\lambda_{c}}\cos\;\theta} \right)}{\delta\left( {{x - {{mx}_{0}\frac{\lambda}{\lambda_{c}}}},{y - {{ny}_{0}\frac{\lambda}{\lambda_{c}}}}} \right)}}}},}}} & {{Eq}.\mspace{14mu} 24}\end{matrix}$

where k=2πn₀/λ, z_(m,n)=(m+Bn)Δz, and x₀=fλ_(c)/d_(x) and y₀=λ_(c)/d_(y)are spatial intervals between consecutive diffraction orders in the xand y directions, respectively on the detector plane for the emissioncentral wavelength λ_(c). A variation of Equation 24 for the oppositefocal shift MFG can also be derived in the similar way by setting thedistortion to be −Δ_(x) and −Δ_(y) in the two directions.

FIG. 11 is a block diagram illustrating a programmed computer system foran interferometric multifocus microscopy imaging system in accordancewith an illustrative embodimedn. In alternative embodiments, othercomputer system architectures, configurations, and/or components can beused to perform the described imaging techniques. The computer system1100, which includes various subsystems as described below, includes atleast one processor 1106 (also referred to as a microprocessor or acentral processing unit (CPU)). The processor 1106 can be implemented bya single-chip processor or by multiple processors. In some embodiments,the processor 1106 is a general purpose digital processor that controlsthe operation of the computer system 1100. The processor 1106 can alsoinclude one or more coprocessors or special purpose processors (e.g., agraphics processor, a network processor, etc.). Using instructionsretrieved from a memory 1107, the processor 1106 controls the receptionand manipulation of input data received on an input device (e.g., imageprocessing, device 1103, I/O device interface 1102, etc.), and theoutput and display of data on output devices such as a display 1101.

The processor 1106 is coupled bi-directionally with the memory 107,which can include, for example, one or more random access memories(RAMs) and/or one or more read-only memories (ROMs). As is well known inthe art, the memory 1107 can be used as a general storage area, atemporary (e.g., scratch pad) memory, and/or a cache memory. The memory1107 can also be used to store input data and processed data, as well asto store programming instructions and data, in the form of data objectsand text objects, in addition to other data and instructions forprocesses operating on the processor 1106. The memory 1107 can also beused to store basic operating instructions, program code, data, andobjects used by the processor 1106 to perform its functions (e.g.,programmed instructions). For example, the memory 1107 can include anysuitable computer-readable storage media that stores computer-readableinstructions that are executed by the processor 1106. The processor 1106can also directly and rapidly retrieve and store frequently needed datain a cache memory included in the memory 1107.

A removable storage device 1108 provides additional data storagecapacity for the computer system 1100, and is optionally coupled eitherbi-directionally (read/write) or uni-directionally (read-only) to theprocessor 1106. A fixed storage device 1109 can also, for example,provide additional data storage capacity. For example, the storagedevices 1108 and/or 1109 can include computer-readable media such asmagnetic tape, flash memory, PC-cards, portable mass storage devicessuch as hard drives (e.g., magnetic, optical, or solid state drives),holographic storage devices, and other storage devices. Mass storages1108 and/or 1109 generally store additional programming instructions,data, and the like that typically are not in active use by the processor1106. It will be appreciated that the information retained within massstorages 1108 and 1109 can be incorporated, if needed, in standardfashion as part of the memory 1107 (e.g., RAM) as virtual memory.

In addition, a bus 1110 to connects the various system components andprovides the processor 1106 access to storage subsystems. The bus 1110can also be used to provide access to other subsystems and devices aswell. As shown, these can include the display 1101, a network interface1104, an input/output (I/O) device interface 1102, the image processingdevice 1103, as well as other subsystems and devices. The imageprocessing device 1103 can include an iMFM system, a camera, a scanner,etc. The I/O device interface 1102 can include a device interface forinteracting with a touchscreen (e.g., a capacitive touch sensitivescreen that supports gesture, interpretation, a microphone, a soundcard, a speaker, a keyboard, a pointing device (e.g., a mouse, a stylus,a human finger), a Global Positioning System (GPS) receiver, anaccelerometer, and/or any other appropriate device interface forinteracting with the computing system 1100. Multiple I/O deviceinterface can be used in conjunction with the computer system 1100. TheI/O device interface can include general and customized interfaces thatallow the processor 1106 to send and more typically, receive data fromother devices such as keyboards, pointing devices, imaging systems,microphones, touchscreens, transducer card readers, tape readers, voiceor handwriting recognizers, biometrics readers, cameras, portable massstorage devices, and other computers.

The network interface 1101 allows the processor 1106 to be coupled toanother computer, computer network, or telecommunications network usinga network connection as shown. For example, through the networkinterface 1104, the processor 1106 can receive information (e.g., dataobjects or program instructions) from another network, or outputinformation to another network in the course of per any of theoperations described herein. Information, often represented as asequence of instructions to be executed on the processor, can bereceived from and outputted to another network. An interface card orsimilar device and appropriate software implemented by (e.g.,executed/performed on) the processor 1100 can be used to connect thecomputer system 1100 to an external network and transfer data accordingto standard protocols. For example, various process embodimentsdisclosed herein can be executed on the processor 1106 or can beperformed across a network such as the Internet, intranet networks, orlocal area networks, in conjunction with a remote processor that sharesa portion of the processing.

In addition, various embodiments disclosed herein further relate tocomputer storage products with a computer-readable medium that includesprogram code for performing various computer-implemented operations. Thecomputer-readable medium includes any data storage device that can storedata which can thereafter be read by a computer system. Examples ofcomputer-readable media include, but are not limited to: magnetic mediasuch as disks and magnetic tape; optical media such as CD-ROM disks;magnetic-optical media such as optical disks; and specially configuredhardware devices such as application-specific integrated circuits(ASICs), program matde logic devices (PLDs), and ROM and RAM devices.Examples of program code include both machine code as produced, forexample, by a compiler, or files containing higher level code (e.g.,script) that can be executed using an interpreter.

The computer system shown in FIG. 11 is but an example of a computersystem suitable for use with the various embodiments disclosed herein.Other computer systems suitable for such use can include additional orfewer subsystems. In some computer systems, subsystems can sharecomponents (e.g., for touchscreen-based devices such as smartphones,tablets, etc., I/O device interface 1102 and display 1101 share thetouch-sensitive screen component, which both detects user inputs anddisplays outputs to the user). In addition, the bus 1100 is illustrativeof any interconnection scheme known in the art that serves to link thesubsystems. Other computer architectures having different configurationsof subsystems can also be utilized.

In summary, described herein is an interferometric multifocus microscopy(iMFM) imaging system. The iMFM imaging system takes advantage ofmultifocus microscopy (MFM) and the configuration with two opposingobjective lenses. The iMFM imaging system provides a higher axialresolution and hence isotropic 3D resolution in a single shot. Thisdisclosure also presents results from examining the problem of combiningI²M and MFM. The new iMFM framework disclosed herein addresses theproblem by employing two diffractive optical elements of opposite focalsteps in the Fourier planes. In this configuration, the emitted lightfrom the same point source can be directed into the same tile on thedetector, and therefore self-interfere after passing through two lenses.The mathematical formulations of iMFM monochromatic and polychromaticPSFs were derived and the image formation models were given. The iMFMPSFs were simulated to show that this new iMFM configuration is capableof recording multiple focal shift interferometry in a single exposurewithout focal scanning, significantly speeding up the acquisitionprocess of conventional detection.

Moreover, two applications of the disclosed iMFM imaging system weredemonstrated: (i) single molecule tracking and (ii) wide field 3Dextended object imaging. The Fisher information matrix (FIM) and theCramér-Rao lower bound (CRLB) of iMFM for both monochromatic andpolychromatic PSFs were calculated. The results show that the iMFM PSFcontains almost uniformly high combined differential information contentalong the optic axis due to the steepened axial features ofinterferometry and the simultaneous multifocal detection scheme, provingthat the iMFM PSF is more effective for encoding a single point positionthan MFM PSF. For 2500 detected photons per objective, a background of10 photons per pixel, MFCTs of 3×3 tiles with a focal step of 0.25 μmand a total efficiency of 67%, and a single camera detection, which aretypical conditions and values in practice, the theoretical localizationprecision of (16.6 nm, 16.6 nm,11.2nm) and (25.6 nm, 24.0 nm, 16.6 nm)in three dimensions can be achieved for iMFM monochromatic andpolychromatic PSFs, with a 4-fold and 3.6-fold axial resolutionimprovement compared with its MFM counterparts. To prevent the lightloss of the BS, a second camera can be introduced and placed at theother output of the BS, where resolution below 10 nm in three dimensionsis obtainable. Another advantage of MFM and iMFM is the focal shiftbetween tiles, which is known a priori and can be used to estimate theaxial initial values of 3D position to improve the accuracy andconvergence of the MLE localization algorithms. The reconstructionerrors found by MLE with a proposed initial value estimation are wellconsistent with theoretical predictions.

For 3D wide field imaging, the dual-objective iMFM tile-OTF featuresabout a 4-fold enlarged support of transferred spatial frequencies inthe axial direction compared with single lens MFM. Therefore, the iMFMimaging system provides improved axial resolution and more isotropic 3Dresolution in wide-field 3D extended object imaging in a single shotwithout focal scanning. It was also shown that even with chromaticaberrations produced from a 10 nm emission bandwidth, the iMFM is stillcapable of recovering high axial spatial frequencies beyond thedetection cut-off. The axial resolution can be further improved if thedual objectives are also used for illumination in addition to collectionas in I⁵M and 4Pi C type microscopes.

Just as in I²M/I⁵M/4Pi dual-objective microscopes, the improved axialresolution in iMFM comes at the cost of system complexity, because twoopposing lenses and two MEGs are utilized for iMFM multifocalinterferometric detection. However, the iMFM imaging system is a singleshot detection without sequentially z scanning focal planes by movingeither the sample stage or the camera, which make the acquisitionprocess not only faster but also much easier and more convenient thantraditional systems. The disclosed iMFM imaging system provides a usefultool in 3D dynamic event imaging in which both high temporal and spatialresolution are required.

The use of the word “a” or “an”, when used in conjunction with the term“comprising” in the claims and/or the specification, may mean “one,” butit is also consistent with the meaning of “one or more,” “at least one,”and “one or more than one.”

As used in this specification and claim(s), the words “comprising” (andany form of comprising, such as “comprise” and “comprises”), “having”(and any form of having, such as “have” and “has”), “including” (and anyform of including, such as “includes” and “include”) or “containing”(and any form of containing, such as “contains” and “contain”) areinclusive or open-ended and do not exclude additional, unrecitedelements or method steps.

Other objects, features, and advantages of the present invention willbecome apparent fro the following detailed description. It should beunderstood, however, that the detailed description and the examples,while indicating specific etribodiments of the invention, are given byway of illustration only. Additionally, it is contemplated that changesand modifications within the spirit and scope of the invention willbecome apparent to those skilled in the art from this detaileddescription.

What is claimed is:
 1. A system to generate an image representations,the system comprising: a first objective that is configured to receive afirst light beam emitted from a sample; a second objective that isconfigured to receive a second light beam emitted from the sample,wherein the first light beam and the second light beam have conjugatephase; a first diffractive element configured to receive the first lightbeam and separate the first light beam into a first plurality ofdiffractive light beams that are spatially distinct from one other; asecond diffractive element configured to receive the second light beamand separate the second light beam into a second plurality ofdiffractive light beams that are spatially distinct from one other; anda detector that receives the first plurality of diffractive light beamsand the second plurality of diffractive light beams, wherein the firstplurality of diffractive light beams and the second plurality ofdiffractive light beams are simultaneously directed and focused ontodifferent portions of an image plane of the detector.
 2. The system ofclaim 1, further comprising one or more lenses and one or more mirrorsthat direct and focus each of the first plurality of diffractive lightbeams and the second plurality of diffractive light beams onto thedifferent portions of the image plane of the detector.
 3. The system ofclaim 1, wherein the first objective and the second objective arepositioned on opposite sides of the sample.
 4. The system of claim 1,wherein each of the first light beam and the second light beam compriseslight from different depth planes within the sample.
 5. The system ofclaim 4, wherein each of the first plurality of diffractive light beamscomprises light from one of the different depth planes within thesample.
 6. The system of claim 5, further comprising a processor incommunication with the detector, wherein the processor is configured togenerate, based on the first plurality of diffractive light beams andthe second plurality of diffractive light beams received by thedetector, a plurality of two-dimensional images corresponding to thedifferent depth planes within the sample.
 7. The system of claim 6,wherein the processor is configured to combine the plurality oftwo-dimensional images to generate a three-dimensional imagerepresentation of the sample.
 8. The system of claim 1, wherein thefirst diffractive element and the second diffractive element arepositioned in a Fourier plane of an optical subsystem of the system. 9.The system of claim 1, wherein each of the first diffractive element andthe second diffractive element comprises a multi-focus grating (MFG),and wherein the multi-focus gratings include a grating pattern having ageometrical distortion.
 10. The system of claim 9, wherein the gratingpattern comprises a binary phase-only diffraction grating or amulti-phase grating.
 11. The system of claim 9, wherein the multi-focusgrating applies at least one phase shift to the first light beam to formthe first plurality of diffractive light beams, wherein the at least onephase shift is equal to but opposite from a depth-induced phase errorthat is present in the first light beam that is emitted fromout-of-focus planes in the sample.
 12. The system of claim 11, whereinthe multi-focus grating applies the at least one phase shift to a firstwavefront in the first plurality of diffractive light beams and to asecond wavefront of in the second plurality of diffractive light beams,wherein the first wavefront and the second wavefront have conjugatephase.
 13. The system of claim 1, wherein the detector comprises anelectron-multiplying charge-coupled device detector.
 14. A method forgenerating image representations, the method comprising: receiving, by afirst objective in a first arm of an optical subsystem, a first lightbeam emitted from a sample; receiving, by a second objective in a secondarm of the optical subsystem, a second light beam emitted from thesample, wherein the first light beam and the second light beam haveconjugate phase; separating, by a first diffractive element in the firstarm of the optical subsystem, the first light beam into a firstplurality of diffractive light beams that are spatially distinct fromone other; separating, by a second diffractive element in the second armof the optical subsystem, the second light beam into a second pluralityof diffractive light beams that are spatially distinct from one other;and receiving, by a detector, the first plurality of diffractive lightbeams and the second plurality of diffractive light beams such that thefirst plurality of diffractive light beams and the second plurality ofdiffractive light beams are simultaneously directed and focused ontodifferent portions of an image plane of the detector.
 15. The method ofclaim 14, further comprising directing, by one or more lenses and one ormore mirrors, each of the first plurality of diffractive light beams andthe second plurality of diffractive light beams onto the differentportions of the image plane of the detector.
 16. The method of claim 14,Wherein receiving the first light beam comprises receiving light fromdifferent depth planes within the sample, and wherein receiving thesecond light beam comprises receiving light from the different depthplanes within the sample.
 17. The method of claim 16, further comprisinggenerating, by a processor in communication with the detector and basedon the first plurality of diffractive light beams and the secondplurality of diffractive light beams received by the detector, aplurality of two-dimensional images corresponding to the different depthplanes within the sample.
 18. The method of claim 17, further comprisingcombining, by the processor, the plurality of two-dimensional images togenerate a three-dimensional image representation of the sample.
 19. Themethod of claim 14, wherein each of the first diffractive elementcomprises a multi-focus grating (MFG) having a grating pattern with ageometrical distortion, and further comprising applying, by the MFG, atleast one phase shift to the first light beam to form the firstplurality of diffractive light beams.
 20. The method of claim 19,wherein applying the at least one phase shift comprises applying a phaseshift that is equal to but opposite from a depth-induced phase errorthat s present in the first light beam that is emitted from out-of-focusplanes in the sample.